Achieving synthesis with meta-analysis by combining and comparing all available studies.

Advances in ecology do not require meta-analysis to answer questions. Yet, what meta-analysis provides is an opportunity to explore why multiple independent tests of these questions can have different outcomes. This exploration is a way meta-analysis can achieve synthesis: by identifying explanations for variation in research while isolating which concepts are applicable over a wide variety of contexts (Glass 1976, Greenland 1994). However, Whittaker (2010) argues—perhaps with some justification based on an audit of multiple conflicting reviews—that a more strict approach to meta-analysis would be more useful for ecology. Specifically, he favors a ‘‘best evidence synthesis’’ that combines both quantitative and qualitative reviewing techniques to answer more narrow questions with only high quality studies (following Slavin 1986, 1994). My intention with this commentary is to explore how stringent the inclusion criteria should be for metaanalysis and the consequences for the breadth or narrowness of the resulting review. I primarily focus on two issues that have received little attention in ecological meta-analysis. First, how the intention and purpose of meta-analysis can impact the scope of the review, and second, how different philosophies on quality assessment can shape the inferences obtained from such reviews. To make these points, I rely heavily on previous discussions from the medical and social sciences about the application of the narrow ‘‘best evidence’’ approach over the broad exploratory alternative. For example, the best evidence approach qualitatively assesses study quality prior to synthesis; whereas the exploratory approach evaluates quality empirically (see Thompson 1994, Eysenck 1995). These opposing philosophies on how quality is treated can significantly alter the rewards of synthesis, perhaps resulting in a review with too few studies to make any useful generalization or a review that lacks the precision to estimate a biologically meaningful effect (van der Velde et al. 2007). Progress in ecological meta-analysis need not develop in isolation from advances in the medical or social sciences, and I hope that by briefly engaging Whittaker’s argument for more narrow reviews with the literature of these fields, I can identify how metaanalysis can be used to validate ecological theory, and why to achieve this goal it is necessary to be broad and inclusive of all available research.